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ABSTRACT 



This study investigated the nature of written instructor 
responses to student writings and the relationship of these written responses 
to the quality of subsequent student work in urban elementary and middle 
schools. Most of the 22 instructors who submitted the study's body of student 
writings (n=114) provided their students with some sort of written feedback, 
and most of their students incorporated that feedback into their subsequent 
drafts. Instructors tended to focus most on standardizing their students' 
written output, with measurable success. Student papers received little 
feedback about content or organization, and these qualities generally did not 
change over successive drafts. Despite the proliferation of reform programs 
targeting urban schools, classroom teachers may need more support to improve 
the skills and knowledge necessary to implement reform practices in the 
teaching of writing. The present study could help focus these reform efforts 
by helping teachers improve the quality of their feedback on student writing 
at both the elementary and secondary level. (Contains 7 tables, 3 figures, 
and 29 references.) (SLD) 
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LEARNING TO WRITE IN URBAN ELEMENTARY AND MIDDLE SCHOOLS: 
AN INVESTIGATION OF TEACHERS' WRITTEN FEEDBACK 
ON STUDENT COMPOSITIONS 



Lindsay Clare and Rosa Valdes 
CRESST/University of California, Los Angeles 

G. Genevieve Patthey-Chavez 
University of California, Los Angeles 



Abstract 

In writing instruction, feedback from teachers provides a critical opportunity for 
students to revise their work and improve as writers. Contexts in which students 
routinely receive feedback on their work include peer reviews and teacher-student 
conferences. For many teachers, however, written comments on student papers remain a 
significant method of response. Despite the importance of teacher responses to student 
work in facilitating the learning process, little research has examined the relationship 
between teacher feedback on early drafts of student work and the quality of students' 
subsequent drafts. Even less research has examined the nature of teachers' written 
feedback to students in K-12 settings. This study investigates the nature of written 
instructor responses to student writings and the relationship of these written responses 
to the quality of subsequent student work in urban elementary and middle schools. Most 
of the 22 instructors who provided the study's corpus of student writings (N = 114) 
provided their students with some written feedback, and most of their students 
incorporated that feedback into their subsequent drafts. Instructors tended to focus most 
on standardizing their students' written output, with measurable success. Student papers 
received little feedback about content or organization, and these qualities generally did 
not change over successive drafts. 

For the past two decades, teachers have been encouraged to adopt a process- 
oriented pedagogy for teaching writing that stresses a cycle of teacher feedback and 
student revision. This approach to writing as a developmental learning process is 
consistent with sociocultural theory, which views development as rooted and 
unfolding in social contexts (Gee, 1990; Heath, 1983; Rogoff, 1990; Schiefflin & 
Gilmore, 1986; Schiefflin & Ochs, 1986; Scollon & Scollon, 1981; Tharp & Gallimore, 
1988; Vygotsky, 1978, 1986). From this perspective, children acquire new skills 
through participation in activities that are supported — or scaffolded — by a more 
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knowledgeable participant. Successful literacy acquisition thus hinges on the extent 
to which novices are given opportunities to engage in meaningful literacy practices 
that build on their funds of knowledge (Moll & Greenberg, 1990) and work within 
students' zones of proximal development (Vygotsky, 1986). 

In writing instruction, teachers' feedback and students' opportunity to revise 
their work based on this feedback are key for this developmental process. Novice 
writers need specific direction and guidance to evaluate, modify, or restructure their 
ideas and to add and delete content to improve the quality of their writing 
(Keppner, 1991; Olson & Raffeld, 1987). Ideally, teacher feedback provides students 
with the opportunity to expand and shape their ideas over subsequent drafts of their 
work (Ferris, 1997; Stemglass, 1998). Through this assisted process, students become 
better writers by gradually appropriating the skills necessary to critically view and 
revise their own work. Zellermayer (1989) quoted Arthur N. Applebee when he 
wrote that learning to write, like all other learning, is 

a process of gradual internalization of routines and procedures available to the learning 
from the social and cultural context in which the learning takes place. Typically new 
skills are learned by engaging collaboratively in tasks that would be too difficult for the 
individual to undertake alone but that can be completed successfully in interaction with 
a parent or teacher. In this interaction the role of the parent or teacher is to provide the 
necessary support, or scaffolding, to allow the child to complete the task, and in the 
process, to provide the child with an understanding of the problem and of the strategies 
available for its solution, (p. 160) 

The goal of this study was to investigate teacher responses on drafts of student 
compositions in urban elementary and middle schools. Contexts in which students 
routinely receive feedback on their work include peer reviews and teacher-student 
conferences. For many teachers, however, written comments on student papers 
remain a significant method of response (Ferris, 1997). While both the social and 
written contexts in which feedback is given are important to study, the specific 
purpose of this study was to investigate the nature of written instructor responses to 
student writings and the relationship of these written responses to the quality of 
student work. 

Despite the importance of teacher response to student work in facilitating the 
learning process, little research has examined the relationship between teacher 
feedback on early drafts of student work and the quality of students' subsequent 
drafts. Even less research has examined the nature of teachers' written feedback to 
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students in K-12 settings. Most of the research in this area has focused on college 
students in general, and more specifically, on college students who are second 
language learners (Ferris, 1997; Keppner, 1991; Olson & Raffeld, 1987; Zamel, 1985). 
Researchers generally have found that teacher feedback during the revision process 
that focuses on the content features of student writings (i.e., encouraging students to 
add and delete content and / or restructure content) as opposed to surface features 
(i.e., word choice, spelling, grammar, and punctuation) is associated with higher 
quality compositions (Keppner, 1991; Olson & Raffeld, 1987). Ferris (1997) similarly 
found that certain types of written comments appeared to lead to more successful 
revisions by college students who are English language learners. For example, she 
found that teacher requests for information, summary comments on grammar, and 
text-specific comments on early drafts of student work appeared to lead to more 
successful revisions, whereas questions or statements that provided information to 
students and less specific comments were less successful. 

Based in part on this body of work and the need for research in this area 
involving younger students, we investigated the following research questions: 

• What is the nature of teachers' written feedback to third- and seventh-grade 
students? 

• Does the type of feedback students receive on early drafts of their 
compositions lead to improvement in the quality of final drafts? 

Importance of the Study 

This study adds important empirical information about a critical instructional 
process in language arts instruction — teachers' feedback to students — and provides 
important information as well about student opportunities to learn in urban school 
environments (National Council on Education Standards and Testing, 1992; Selden, 
1988; Shavelson, McDonnell, & Oakes, 1989). Despite a proliferation of reform 
programs targeting urban schools over the past 20 years, there has been little change 
in the academic achievement of poor and minority students, who continue to 
perform well below state and national norms on standardized tests of achievement. 
According to the National Commission on Teaching and America's Future (1996), 
many teachers have difficulty achieving instructional goals set out in many reform 
programs because they do not have the knowledge and skills needed to significantly 
improve the quality of the learning environments they create for students. This 
problem is especially pronounced in urban schools, where there tend to be fewer 
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experienced teachers and greater numbers of students with special learning needs 
(National Commission on Teaching and America's Future, 1996). The present study 
could help focus reform efforts by drawing attention to this critical aspect of practice 
and providing meaningful information to teachers about ways to improve writing 
instruction in their classrooms. 



Methods 

Written feedback from teachers was investigated with a corpus of 114 pieces of 
student writing, including rough and final drafts from urban third- and seventh- 
grade classrooms (50 pieces of elementary student writing and 64 pieces of middle 
school student writing). This corpus was collected in 10 schools (five elementary and 
five middle schools) over a two-year period as part of a larger study evaluating the 
influence of a large-scale school reform initiative on the quality of students' learning 
environments ( N = 11 elementary school teachers and N — 11 middle school 
teachers). Teachers' work experience ranged from approximately 2 to 28 years. The 
schools in our study served primarily minority students who are English language 
learners. The percentage of children in our sample schools receiving free or reduced- 
price lunches ranged from 56.6% to 96.4% (see Table 1). 

Procedures and Measures 

Teachers were asked to provide us with four language arts assignments — 
"typical" writing with multiple drafts, "typical" reading comprehension, content 



Table 1 



Demographics for Elementary and Middle Schools (N = 10) 





Elementary schools ( n = 5) 


Middle schools ( n = 5) 


% 


% Range 


% 


% Range 


Enrollment by ethnicity 










Asian 


7.3 


0.3 - 27.0 


7.1 


0.7-22.4 


African American 


9.9 


1.1-20.0 


14.4 


0.6-35.5 


Latino 


68.5 


34.0-92.1 


64.1 


41.0-92.7 


White 


11.7 


0.5 - 37.3 


10.2 


0.3-31.9 


Other 


2.4 


0.3 -7.0 


4.2 


0.6 - 17.0 


English language learner 


69.1 


50.4-82.2 


44.0 


30.0-602 


Free/ reduced lunch 


91.2 


86.7 - 96.4 


75.5 


56.6 - 86.9 
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area writing (elementary school only), and a "challenging" major project. Teachers 
also were asked to complete a one-page information sheet for each assignment and 
to submit four samples of student work for each assignment: two that they 
considered to be of "medium" quality and two of "high" quality. The teacher 
assignment materials (notebook, cover sheets, consent forms, etc.) were distributed 
in the fall and collected in the winter and spring. 

In this study we focused on the type and quality of feedback offered on the 
typical student writing assignment only (a final writing project with earlier drafts). 
Assignments that did not include rough drafts of student work were not included in 
the analyses. The personnel who had given students feedback in classroom 
conferences were categorized based on teachers' self-report on their assignment 
cover sheets. The type of writing elicited (informative, persuasive, narrative) and the 
academic nature of the writing (i.e., whether the assignment required students to 
interact with content knowledge [including knowledge of literature] also were 
categorized. The number of words in student writings was included as an indicator 
of writing fluency and quality (Patthey-Chavez & Clare, 1996; see Table 2). 

Table 2 . 



Descriptive Information about Assignments and Student Work (N = 114) 





Elementary school 
( n = 50) 


Middle school 
( n = 64) 


Type of writing elicited 






Informative 


20% 


59% 


Persuasive 


— 


19% 


Narrative 


80% 


22% 


Academic content knowledge elicited 






None 


88% 


33% 


Other discipline-based 


4% 


20% 


Literature-based 


8% 


47% 


Mean number of words in students' essays 
Persons who gave feedback during writing process 


133 (SD = 98) 


270 (SD = 172) 


Teacher only 


8% 


29% 


Peers only 


— 


3% 


Teacher and peers 


80% 


51% 


Teacher, peers, and others 


12% 


5% 


Nobody 


— 


12% 
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Following Olson and Raffeld (1987), one researcher categorized the type of 
feedback students received on each draft as "surface level" or "content level." 
Student papers that received no written comments were noted as well. Reliability of 
the categorical designations was assessed by having a second researcher 
independently score 20% of the assignments chosen at random. Interrater agreement 
for these scales was 80%. The amount of written feedback students received on their 
drafts was determined by obtaining a ratio of the number of edits and comments a 
student received divided by the number of words in the composition. 

Drafts of student writings were rated by a bilingual rater using three 
standards-based scales measuring organization, content, and writing mechanics, use 
of language, grammar, and spelling (MUGS). These scales are from the Language 
Arts Project rubric developed by CRESST at UCLA in partnership with the Los 
Angeles Unified School District and United Teachers-Los Angeles (Higuchi, 1996). 
Each of these dimensions was rated on a 4-point scale (1 = poor, 4 = excellent). 
Reliability of these ratings was assessed by having another bilingual researcher 
independently score 20% of the assignments chosen at random. Overall exact scale- 
point agreement for the student work scales was 81%. 

Analyses 

Descriptive statistics and frequencies were computed to examine the quality of 
student writing and type of feedback teachers gave to students on their written 
work. Correlation coefficients were used to investigate the relationship between 
teacher feedback and the quality of student work, and t tests for paired samples 
were conducted to explore changes in the quality of student work from earlier drafts 
to their final draft. Regression analyses were then used to investigate the relative 
influence of teacher feedback on the quality of final drafts of student compositions. 
Finally, a more qualitative approach was adapted from earlier studies tracking 
longitudinal changes in student writings across multiple drafts (Ferris, 1997; 
Patthey-Chavez. & Clare, 1996; Patthey-Chavez & Ferris, 1997). This involved 
classifying and then tracking particular teacher recommendations from draft to draft 
to see whether they were taken up and incorporated into student revisions. This 
final analysis adds important information about teacher feedback in the K-12 system 
to the typology of teacher responses developed to date. 
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Results 



To address our first research question concerning the nature of teachers' 
written feedback to students, we examined the amount and type of feedback 
students received on drafts of their compositions. As illustrated in Table 3, our 
results indicate that the students by and large had limited opportunity to receive 
content-level feedback on their writing in both elementary and middle school. Fifty- 
eight percent of the students received only surface-level feedback on drafts of their 
compositions. This trend was more pronounced in third grade, where fewer 
students received any kind of written feedback and less content-level feedback was 
offered than in seventh grade. Students in middle schools received slightly more 
feedback than the elementary school students as evidenced by the ratio of edits per 
number of words in the student work samples (see Table 3). 

To investigate our second research question concerning the influence of 
teachers' written feedback on the quality of student work, we first looked at the 
difference in the length of student essays from the preliminary to the final drafts 
written by those students who received surface-level versus content-level written 
feedback. We found that middle school students who received content-level 
feedback increased the length of their essays from the preliminary to the final drafts 
by an average of 48.14 words (see Table 4). Middle school students who received 
only surface-level feedback, in contrast, increased the length of their essays by an 
average of 16.86 words. Elementary school student essays did not increase in length 
on the whole from the preliminary to the final drafts. In fact, student essays that 
received only surface-level feedback tended to decrease in length by an average of 

Table 3 



Type and Amount of Written Feedback on Student Essays 





Elementary school 
(n = 50) 


Middle school 
(n = 64) 


Type of written feedback 






None 


28% 


8% 


Surface level 


58% 


58% 


Content level 


14% 


34% 


Amount of written feedback 






Average ratio of edits to total number 
of words per student work sample 


.08 (SD = .08) 


.04 (SD = .04) 
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Table 4 



Type of Written Feedback and Change in the Number of Words in Student 
Essays From Preliminary to Final Drafts 





Elementary school 
M (SD) 


Middle school 
M (SD) 


Type of written feedback 






Surface level ( n = 66) 


-4.72 (21.98) 


16.86 (37.25) 


Content level ( n = 29) 


0.00 (27.47) 


48.14 (61.82) 



4.72 words. While the small sample precludes any statistical analysis, this finding 
concurs with other research on the importance of providing content-level feedback 
to fledgling writers. 

We then used regression analytic techniques to further investigate the influence 
of teachers' written feedback on the quality of student work. These analyses 
revealed that when the quality of first drafts was held constant, the type of feedback 
students received did not influence the quality of the final drafts of student work 
(see Tables 5 and 6). The amount of teachers' written feedback predicted 
improvement in student writing with regard to mechanics (i.e., punctuation, 
grammar, and spelling) at both the elementary school ( b = 2.61, R 2 = .14, p < 0.01) 
and middle school ( b = 3.24, R 2 = .13, p < 0.01). Given the relative prevalence of 
instructor feedback on mechanics, this result is hardly surprising. In essence, 
students appeared to respond to the actual feedback they received. 



Table 5 

Results of Multiple Regression Analysis for Elementary School Student Work (n = 50) 



Outcome score 




Content 


Organization 


MUGS 


Predictor 


b p R 2 


b p R 2 


b (1 R 2 


Standardized amount of feedback 
(ratio) 

Type of feedback 
(none / surface / content) 






2.61** .38** .14 



Note. MUGS = Mechanics, Usage, Grammar, Spelling. 
*> < 0.01. 
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Table 6 



Results of Multiple Regression Analysis for Middle School Student Work (n = 64) 



Outcome score 




Content 


Organization 


MUGS 


Predictor 


b |JR 2 


b 3 R 2 


b 3 R 2 


Standardized amount of feedback 
(ratio) 

Type of feedback 
(none / surface / content) 






3.24** .36** .13 



Note. MUGS = Mechanics, Usage, Grammar, Spelling. 
*>< 0 . 01 . 



To further investigate the relative influence of the type of feedback students 
received on drafts of their compositions, we examined the degree to which student 
compositions changed in quality from early drafts to the final draft. The results of 
this analysis indicate that elementary school students' compositions on the whole 
improved with regard to the quality of writing mechanics (MUGS) from the early 
draft to the final draft (t = 3.90, df = 49, p < 0.001), but not with regard to the quality 
of the content or organization. Middle school students made very small gains overall 
in the quality of their essay content (t = 2.18, df= 63, p < 0.05) and organization ( t = 
3.21, df= 63, p < 0.05) (see Table 7). This minimal improvement may be attributed in 
part to the lack of quality feedback students received. As described earlier, only 34% 
of the feedback provided on middle school essays and 14% of the feedback provided 
on elementary school essays addressed content-level issues (see Table 3). Our 
findings suggest that the lack of improvement in the quality of students' essays 
reflects, at least in part, the absence of quality feedback on early drafts of their 
writing. Again, students appeared to respond to the type of feedback they received. 



Table 7 

Mean Difference Scores From Early to Final Drafts of Student Work 





Elementary school 
(n =50) 






Middle school 
(w = 64) 




M 


SD 


Min. 


Max. 


M 


SD 


Min. 


Max. 


Content 


.02 


.14 


0 


1 


.09* 


.34 


-1 


1 


Organization 


.06 


.24 


0 


1 


.14* 


.35 


0 


1 


MUGS 


.30*** 


.54 


-1 


2 


.08 


.37 


-1 


1 



Note. The scale for each item ranged from 1 to 4. 
><0.05. **><0.001. 
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Summary of Results From the Statistical Analyses 

In summary, the quality of student writing showed relatively little 
improvement over successive drafts, and students received little by way of content- 
level written feedback on early drafts of their writing. Most of the feedback students 
received pertained to improving punctuation, grammar, and spelling, and students 
responded by improving punctuation, grammar, and spelling. On the whole, middle 
school students who received content-level feedback had a greater increase in the 
length of their essays from the preliminary to the final drafts than middle school 
students who only received surface-level feedback. The length of the elementary 
school students' essays, in contrast, basically remained the same regardless of the 
type of feedback they received. When the quality of students' early drafts was held 
constant, however, the type of written feedback students received did not predict 
improvement in the quality of student work as assessed on a standards-based rubric 
at both levels of schooling. The amount of feedback students received (comments 
and edits) predicted improvement in the quality of writing mechanics only. 

Exploring Teachers' Written Feedback and Student Essays: A Qualitative 
Perspective 

To better understand our findings, we returned to our corpus of essays to 
investigate more closely the relationship (or lack of relationship) between the type of 
written feedback students received and changes in the quality of their essays over 
successive drafts. As illustrated in Table 3, most of the students in our sample 
received only surface-level feedback. The following drafts of a seventh-grade 
student's essay provide a good example of the type of written feedback students 
received, as well as of the lack of substantive improvement in the quality of their 
work. In the case of this writer, all teacher feedback addressed surface-level issues. 

This essay, considered by the teacher to be of relatively high quality for the 
class, describes the qualities of a hero. Comparing the first draft with the fourth and 
final draft, we see that some words were changed (e.g., being "respective [sic] " was 
changed to being a "role model") and a few organizing phrases were added (e.g., 
"these are the requirements of being a hero"). Substantive improvement in quality, 
however, only occurred in the area of spelling and punctuation. In fact, it is notable 
how little the essay changed from the first to the fourth draft. The teacher made a 
number of edits on each draft focusing on correcting spelling, word usage, 
punctuation, and grammar, and the student appeared to have copied most of these 
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teacher edits from draft to draft. This is likely why the essay improved in the quality 
of writing mechanics (and why the amount of feedback predicted some 
improvement in quality for middle school student essays). Clearly, however, the 
student was not provided with the guidance that would have been necessary to 
substantively develop the ideas expressed in the composition (see Figure 1). 



Draft 1 

What is a hero? A hero is someone who would sacrife any thing and would have to be 
someone who would influene you not in a bad but in a good way. Heroes would have be very 
tolerant too. A hero would also be very respective and most of all would have a good soul. 

A hero most of the time is very persistant. He would never give up no matter what anyone 
said. They might have a lot of very putdown's, but that would never stop him. When a hero 
reaches his goal he is most of the time admired or celebrated. 

Most of the time, when a hero has reached his goal, he influence's many other people. 
Sometimes by the influence of that hero he gives rise to anew hero. Or sometimes if that hero has 
an enormuse positive self-essteem he might influence him self to do another positive act or event. 

One of the most important thing you have to do to be a hero is be a role-model. I have 
never seen a hero that is a no mannered, disrespectful person for an example my personal hero 
is my dad. He is a perfect rolemodel & I have never seen him drunk or disrespecting any one. 

Draft 4 (final) 

What is a hero? A hero is somebody who would be persistent & do anything to help you. It 
would be necessary that he would influence you not in a bad But in a good way. Also every 
hero would have to be a role model. I think these are the requirements to be a hero. 

A hero most of the time is very persistent. He would never give up, even though the battle 
is all uphill. They might get a lot of putdown's but would not be enough to stop them. When a 
hero reaches his or her goal, he is most of the time admired or celebrated. 

Most of the time when a hero reaches his goal, he influences other lives. Sometimes if that 
certain hero has an enourmuse amount of self-esteem he might influence himself to do another 
positive act or event. 

One of the most important requirements you need to have to be a hero is to be a role . 
model. I have never seen a no mannered, disrespectful person that was ever called a hero. For 
example, one of my personal hero's is my dad, because I have never seen him drunk or 
disrespecting anyone & that makes him a perfect role-model. 



Figure 1. Seventh-grade essay that received surface-level feedback only. 
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We wondered why content-level written feedback also did not lead to 
substantive improvement in the quality of student work. To explore this issue 
further, we took a closer look at the nature of the content-level feedback students 
received on their essays and the degree to which students incorporated this 
feedback into the final drafts of their essays. 

Overall we found that all of the content-level feedback elementary students 
received and about half of the content-level feedback received by middle school 
students were requests of some type to add clarifying information. For example, 
after one third-grade student wrote "They went to sleep and had nightmares," the 
teacher inserted "about what?" In a seventh-grade student's essay, the student wrote 
"We use twice as much water"; the teacher circled the word "we" and wrote next to 
it "Who is we?" and wrote "meaning?" after other sentences. 

To investigate the degree to which students incorporated their teachers' 
feedback, we traced the specific feedback students received on early drafts to their 
final drafts. For the elementary school essays we found that only three out of seven 
student essays showed evidence of students incorporating their teachers' request to 
add or clarify information. For example, one student's essay that showed evidence 
of incorporating the teacher's feedback was a story called "The Teacher from Outer 
Space." On the preliminary draft the student wrote "They went to bed and had 
nightmares." The teacher circled "nightmares" and wrote "about what?" In the final 
draft, the student wrote "They went to bed and had nightmares about the teacher 
taking off her skin." Another student for that same assignment, in contrast, was not 
able to successfully incorporate the teacher's feedback in the final draft of his story, 
"Shark Enemies." In the first draft this student wrote "One day there were another 
army that was nicer." The teacher circled "army" and wrote "of who?" In the final 
draft of the student's story the descriptor "nicer" was changed to "friendlier," but 
the student did not clarify who constituted his story's army. The sentence remained 
virtually unchanged: "One day there were another army that was friendlier." 

The following drafts of an essay written by a third-grade student, considered 
by the teacher to be of high quality for the class, illustrate typical content-level 
written feedback in our sample and attendant influence on student work within our 
small corpus of elementary school student essays with such feedback (see Figure 2). 
The teacher's goals for the writing assignment included the requirement that 
students learn to write in a "report of information" style. The content-level comment 
on the following first draft of a student's report about sharks was a request to add a 
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Draft 1 



Sharks have sharp teeth and they have clod booled. Some sharks live in undergourd and they 
eat fish and seals so they could sravey. I they are cool shark. They could grow from twenty feet. 
They are bad because there are different kinds of sharks. I saw a show about sharks that a man 
shark could pull your legs. Shark could swim fast as people there are lot of shark at the ocean 
and shark like to fight other for fish since sharks live in Mediterranean sea the biggest shark is 
the whale shark there are the meber shark of family the baby was caught in Florida they could 
weighed over 15 tones. Sharks live in warm water. 



Draft 2 (final) 

The Sharks 

Sharks have sharp teeth and they are cold blooded. They breath through gills. Sharks live near 
reefs in the ocean. They eat fish and seals so they can survive. They are dangerous. They could 
grow from twenty feet. They are mean because there are different kinds of sharks. Some sharks 
can bit your legs you could die. Sharks could pull your legs. Sharks can swim fast as people. 
There are lot of sharks in the ocean. Some sharks like to fight other sharks. They fight for fish 
some sharks live in Mediterranean sea the biggest shark is the whale shark. There are other 
members of the shark family. The baby was caught in florida. They could weigh over fifteen 
tons. Sharks live in warmwater. 



Figure 2. Third-grade student essay that received content-level feedback. 



title and "more facts about sharks." The teacher also changed the student's 
statement that sharks live "underground" to sharks "live near reefs in the ocean." 
The student's initial description of sharks as "cool" also was changed by the teacher 
to "they are dangerous," and "bad" was changed to "mean." Most of the written 
feedback concentrated on spelling, grammar, punctuation, and word usage. These 
changes are evidenced in the final draft of the essay, along with the added fact that 
sharks breathe through gills. 

As shown in Figure 2, the student's punctuation and sentence structure 
changed quite noticeably towards the written standard. And in keeping with the 
assignment's goal, that of writing an information report, more personal notes from 
the first draft — "I they are cool shark [sic]" and "I saw a show about 
sharks" — disappear and are replaced by a few "more facts about sharks" and a title, 
"The Sharks." As in a number of other information reports observed by the third 
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author, the essay is reminiscent of an encyclopedia and amalgamates rather than 
summarizes facts, such as in the somewhat illogical "They are mean because there 
are different kinds of sharks." It is perhaps the impersonal origin and approach that 
account for this rather haphazard reporting. And for the most part, the organization 
and content of the essay remain unchanged from the first to final drafts. 

For our corpus of seventh-grade essays that received content-level feedback 
( n = 22), approximately two thirds of the students incorporated the teachers' 
comments into the final drafts of their essays (n = 15). Whether there was a 
substantive change in the quality of the essay, however, appeared to depend on the 
nature of the feedback the student received in tandem with a student's response. 
Only half of the seventh-grade essays ( n = 10) received content-level feedback that 
was text-specific and appeared to provide students with at least some guidance for 
how to substantively revise their work. Of these 10 essays, eight of the final drafts 
showed evidence of having incorporated teacher feedback. 

We looked at two student essays in particular that received text-specific 
content-level feedback from the teacher. We believe that in both cases, the feedback 
offered the students an opportunity to make substantive revisions to their work. The 
students' assignment was to analyze an article from an environmental magazine and 
write a letter to the appropriate agency in which they identified the causes of the 
problem and posed a workable solution. One student wrote in his preliminary draft 
"What you can do to help support efforts to stop the poaching of mountain tapirs 
and ensure their protection in Ecuador you can contact Wildlife Conservation 
International." In response to this, the teacher wrote "You need to pose a solution 
and show how it might work." In the final draft, the student added four suggestions 
for saving the tapirs including "What we can do to stop this is to divide the land, 
half for the hunters and half for the mountain tapirs." Another student with the 
same assignment, in contrast, did not (or was not able to) incorporate the teacher's 
feedback into his final draft. The teacher on the preliminary draft of this essay asked 
the student to "explain about wood," and to clarify whether the student was 
referring to "the government or U.S. companies?" The student did not clarify this 
point or provide additional information in his final draft. 

The following is an additional example of an essay that was substantively 
revised between the preliminary and the final drafts in response to the teacher's 
content-level feedback. In this assignment the teacher asked the students to analyze 
the poem "Alone," by Maya Angelou, by "explaining the speaker's thoughts while 
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connecting the poem to their knowledge of the world." Students were then asked to 
revise their first drafts (a response to a writer's prompt) into a "well-organized 
essay" (see Figure 3). 

On the first draft the teacher wrote a general comment to the student, 
reminding him, "You need four paragraphs. Quote poem and explain. Connect 



Draft 1 

The speaker of this poem seems to be mentally suffering from how this poem is put 
together and the message it's sending. The speaker believe that the writer of this poem express 
her feelings and emotion into this poem. She thinks that no one could love alone and the 
speaker knows because a person needs a friend or a companion to be with or talk to. 

Personally, I think that the message of this poem is to always be good to others in order to 
have a friend for companionship. You need someone to express your feelings to. I would 
definitely agree with the speaker of this poem because it is true and a person without a friend is 
a nobody. 

Draft 2 (final) 

The speaker of this poem seems to be mentally suffering from this poem. The speaker 
expresses their feelings into this poem and describing the place that they live in. "The race of 
man is suffering" which the speaker describe is their world. I think it means that they can't 
work together and compete with each other. 

The speaker knows this because some people are very poor and millionaires are so selfish 
that they don't think money are to be use on them. Personally, I think that the speaker is tired 
and sick of their world. I would say that this is like our world because wars are going on and 
many people are living on the street. "And I can hear the moan" describes my reason of people 
who don't have a home. 

I would say that the main theme of this poem is in your world, start over and let 
everybody have a chance in life that they have. You should care for others and share your 
wealth with unfortunately. "There are some millionaires. With money they can't use" are 
people who are greedy and the message is the opposite because you can use the money to help 
others. 

I would definitely agree with the speaker of this poem because our world is nothing 
different from it. People are fighting for land and nobody seems to share it. The race of man is 
suffering is my opinion of everything. The repition helps to say that you need a friend and not 
fight. 



Figure 3. Seventh-grade student essay that received content-level feedback. 
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poem with outside world." As illustrated in the preceding essay drafts (see Figure 
3), the student responded to each of the teacher's comments. The final draft of the 
essay was expanded from two to four paragraphs, and the student quoted from the 
poem and connected the content of the poem to the outside world by referencing 
society's problems, such as homelessness and war. While clearly there still are 
problems with the essay's syntax, expression of ideas, and general readability, the 
final draft reflects the writer's additional engagement with the poem and growth in 
terms of the sophistication of his essay's content. 

Seventh graders writing 9 of the 22 essays that received content-level feedback, 
or less than half, did not get an opportunity to substantively revise their work. As 
with the elementary school essays, this type of content-level feedback only required 
students to reword sentences to be clearer and specify pronoun usage. Of these 9 
essays, 7 showed evidence of having been revised per teacher instructions. Non- 
substantive content-level feedback from the teacher was usually incorporated with 
little change in the nature or quality of the final draft. In one assignment, the teacher 
asked students to write a letter to another student recommending their favorite 
book. In the early draft of his letter, one student wrote "This cunning quote is 
uttered by the character in an excellent novel about Native Americans." The teacher 
added the character's name, so that this sentence in the student's final draft read 
"This cunning quote is uttered by the character. True Son, in an excellent novel 
about Native Americans." The student later wrote "the [sic] passages are authentic 
phrases said by the Indians." The teacher circled "said" and wrote "stronger verb" 
next to it. In the final draft, this sentence read "the [sic] passages are authentic 
phrases uttered by the Indians." This was the extent of the content-level feedback 
the student received. Not surprisingly, as with the student referenced in Figure 1 
who received only surface-level feedback, this student's final draft was nearly 
identical to his early draft. 

Finally, three essays received substantive content-level feedback on the final 
drafts only. While this feedback appeared to be of high quality, the fact that it was 
written on the essays' final drafts likely means that these students did not have an 
opportunity to make use of this feedback. While it is possible that the teachers might 
have provided these comments in a conference with the students prior to the 
submission of their final drafts, it appears to us that this feedback came too late in 
the writing process to be of use to these students. 
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Summary of Qualitative Findings 

In summary, our qualitative investigation of teachers' written feedback and 
student essays confirmed our statistical analyses. As expected, the surface-level 
feedback students received provided them with an opportunity to fix their 
grammar, punctuation, and spelling errors but not the content and organization of 
their essays. Not expected by us was our finding that the content-level feedback 
students received also, by and large, did not provide them with much of an 
opportunity to substantively revise their work. The small percentage of elementary 
student essays that received content-level feedback (14% of 50 essays, or 7) only 
asked students to clarify pronouns and other references in their writing. The middle 
school essays that received content-level feedback (34% of 64 essays, or 22) were 
divided in terms of whether students received at least some guidance on ways to 
substantively revise their work. In our view, approximately half of the essays 
received text-specific feedback that provided this type of guidance ( n = 10) and 
nearly half received content-level feedback that, as with the elementary school 
essays, only provided students with an opportunity to clarify their pronouns and 
other contextual references. A very small number of essays ( n = 3) received content- 
level feedback on the final drafts only. Considering this, it is not surprising that the 
type of feedback students received predicted improvement in writing mechanics 
only, as opposed to content and organization (see Tables 5 and 6), and that student 
work showed only the barest improvement across drafts (see Table 7). 

Summary, Conclusions, and Directions for Future Research 

Returning to our research questions, most of the written feedback students 
received on drafts of their compositions was on the surface level in that it pertained 
to improving punctuation, grammar, and spelling only. Students received little in 
the way of content-level written feedback on drafts of their writing, especially in the 
earlier grades. This finding is commensurate with other research that has found that 
writing teachers frequently attend more to how things are written and less to the 
content and structure of students' ideas (Beason, 1993; Hopman & Glynn, 1988). 

We then investigated whether or not teachers' written feedback predicted 
improvement in the quality of student work in elementary and middle schools. Our 
analyses indicated students who received content-level feedback on early drafts of 
their writing were more likely to increase the length of their final drafts than those 
students who only received surface-level feedback. When the quality of students' 
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early drafts was held constant, however, the type of written feedback students 
received did not predict improvement in the quality of student work. In other 
words, higher quality first drafts turned into higher quality final drafts and lower 
quality first drafts turned into lower quality final drafts, with little guidance from 
instructors and no statistically significant relationship to teacher commentary. The 
very scarcity of that teacher commentary predicts such an outcome, and it 
should — and does — raise concerns about the meaning of a process approach that 
mostly provides students with multiple opportunities to recopy rather than revise 
their work. The amount of feedback students received (the number of comments and 
edits) significantly predicted improvement in the quality of writing mechanics only, 
though the percent of variance explained was relatively small. On a more qualitative 
note, there is a definite trend towards more standardized language use. 
Standardized language use was clearly attended to by most of the instructors in our 
study, since 58% of their comments aimed to standardize some aspect of 
punctuation, spelling, or grammar, and that focus was reflected in a decreasing 
number of errors. Middle school students who received content-level feedback also 
added more to their essays (as measured by an increase in the number of words) 
than middle school students who received surface-level feedback only. This trend 
was not apparent in our corpus of elementary school student essays, however. 

Since the process approach to writing instruction aims to not only standardize 
writing but to improve it as well, we took a closer look at the nature of the content- 
level written feedback received by students. All of these written comments at the 
elementary school level and about half of these comments at the middle school level 
were requests for students to clarify the meaning of a particular word or sentence. 
Generally speaking, these comments were fairly generic in nature and were not 
explicitly focused on helping students reflect on or substantively reorganize their 
ideas. Requests to clarify pronouns or "add more facts about sharks" and comments 
such as "awkward, missing topic sentence," for example, do not invite much 
reflection on the subject matter or attendant revision. 

From a sociocultural perspective, it appears that teachers' written feedback on 
successive drafts of the essays by and large did not create an opportunity for the 
kinds of collaborative assisted interactions central to learning. Students copied the 
corrections in spelling, grammar, and punctuation their teacher made on their 
papers; they may even have been prodded to draw from other sources, as our third- 
grade essay about sharks suggests. But on the whole, students were not asked to 
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engage in genuine revision, and they did not do it on their own. As a result, the 
quality of student writing showed relatively little improvement over successive 
drafts. The process approach to writing instruction may have become the official 
policy in these schools, but its execution does not differentiate it much from earlier 
approaches to writing instruction. We doubt whether the old "product" approach 
with its emphasis on error correction could reliably be distinguished from the 
particular "process" approach implemented by our instructors on the basis of these 
data. 

Our research provides another example of an instructional practice advocated 
by reformers without much consideration for how to implement it in the classroom. 
Teachers do not necessarily have the knowledge and skills to implement the practice 
(in this case the writing process) as it was originally conceived. This problem in 
implementing new instructional practices has been noted by other researchers 
(Cohen & Ball, 1994; Spillane & Zeuli, 1999). For example, Cohen and Ball found that 
even those teachers who actively endorsed a curriculum innovation for teaching 
mathematics and who claimed to be implementing the innovation in their 
classrooms did so incorrectly or only partially for the most part. Isolated in their 
classrooms, teachers are generally left to interpret new approaches to teaching 
through the lens of their own professional experience. Cohen and Ball concluded 
that teachers need much more support for implementing reform practices than is 
generally offered to them in traditional professional development activities. The 
same point was made rather thoroughly by Tharp and Gallimore (1988) and by 
McNiff and Leader (1995) and a number of other Action Researchers. Furthermore, 
the translation of an instructional practice, like the process approach, into a set of 
measurable standards is difficult (Dyson, 2000). 

Reform programs seeking to improve students' literacy skills should include a 
substantive focus on helping teachers improve the quality of their feedback on 
student writing at both the elementary and secondary level. Students do not grow as 
writers in the absence of high-quality feedback, and neither do teachers. As with 
students, teachers need opportunities for collaborative assisted professional 
development to grow as instructors and create more effective learning environments 
for students. They need models of good feedback to students, and they need to see 
examples of appropriately revised work at each grade level. Our data suggest that 
teachers are sensitive to the importance of standardizing their students' written 
language, a primary emphasis of many standardized assessments used to evaluate 



students and schools. They may not be paying as much attention to their students' 
use of writing as a means to develop and communicate ideas. 

Future research could assist this process by looking more deeply at the types of 
written feedback that do improve the quality of student writing so that grade- 
appropriate models can be generated and shared with teachers. Our research 
reported here was limited by a small sample size, and may have been limited as well 
by the general categorical nature of our feedback designations (i.e., surface level 
versus content level). For example, Ferris (1997) looked at finer categories of written 
teacher responses to college students who are English language learners and found 
that text-specific feedback was more helpful than "vague prescriptive remarks" on 
student papers (p. 333). Future research should likewise look more deeply at the 
nature of effective written feedback for younger students and attempt to categorize 
it. 
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